posted 01-14-2011 10:46 AM            

quote:

---

1. I have read that ESS scoring rules are for single-issue tests (Blalock, Cushman, Nelson, 2009). Can we score multi-facet test (for example Utah ZCT with three different questions) with the ESS as single-issue test if we know that a subject must answer all of the questions truthfully or all deceptively? Or it is another decision model?

2. Do I understand correctly that with the OSS-3 (using two-stage rules) and with the ESS (-7 for spot)
we can make the DI decision with significant reaction to only one question in the signle-issue test despite that in such test a subject must answer all of the questions truthfully or all deceptively?

3. Do I understand correctly that with the OSS-3 we can make the DI or NDI decisions in the multi-facet test using grand mean rules despite that a subject could be deceptive to one question while being truthful to others or vice versa?

4. Are there differences for the OSS-3 between the single-issue test and multi-facet test with using grand mean or two-stage rules? When I change a type of the test from the single-issue to multi-facet there is no differences in results.

Thanks.

[This message has been edited by skar (edited 01-14-2011).]

---

Skar,

Good questions. Too much for one sitting.

Keep in mind that there is just about no research that actually supports the multi-facet hypothesis. These are, in fact, event-specific (a better term than single-issue because you can have a single issue screening test) examinations.

I was once discussing the fact that the single-issue three-question ZCT, such as the Utah or Federal versions, seem to be the most accurate diagnostic techniques we have. Charles Honts corrected me and stated that the Utah ZCT been researched and validated as a multi-facet exam - to which I pointed out that they scored it by grand-total as a single-issue. Honts agreed.

Then look at the studies by Senter and Dollins (2008), and earlier studies, and you will see that the use of two-stage rules results in better criterion accuracy.

IF the multi-facet hypothesis were true - THEN we would have to see some form of improvement in criterion validity from the construction and interpretation of the multi-facet-ness of the test questions.

Improvement, for a scientific test would have to be observed in the form of increased criterion validity: increased sensitivity, increased specificity, decreased FP or FN errors, or decreased inconclusives.

Where is that evidence in the form of published studies, not just some expertized optinion (untested hypotheses) or case annecdotes that have imprinted someone's perceptions?

The evidence tells us that the multi-facet idea does increase test sensitivity (not specificity), and does decrease FN errors. The cost is very weak (worse than chance) test specificity), high inconclusives, and high FP errors (though this can be managed through statistically optimal cutscores in OSS-3 and ESS).

However, the evidence seems to suggest that it is the decision rules and cutscores, and NOT the fancy linguistic precision of the test questions that is contributing to the change in criterion accuracy with multi-facet and multi-issue exams. Common criminals and psychopaths are linguistically imprecise creatures... It is unlikely that our nearly-OCD interest in semantic precision will actually connect with or impress their broken brains. Linguistic precision is for the the truthful person - to attempt to eliminate causes for reaction other than deception regarding involvement in the factual behavioral concern described by the test question.

When you require that the examinee "pass" all questions, you also have to define what you mean by "pass." Do you mean the examinee must produce a statistically significant truthful score to each question? If so, you have to state your required level of significance or tolerance for error. You must also have a way of calculating that-there probability of error. OSS-3 and ESS give you that ability.

Also, when you require the examinee to "pass" every question, you are engaging a multi-facet or multi-issue testing paradigm.

If you read the published studies, you will see that the polygraph is accurate at the level of the test as a whole. In other words, people pass and fail the the test, not the individual questions.

This, of course, does not tickle our desires for a super-duper-magical-wonder-test that can do everything we want, every time, and never make a mistake. But guess what? There is no such thing as a perfect test in any field of science!

Well, to be sure, there are people who will tell you that they have a super-duper-mind-reading-wonder-test, that they have it all worked out and they have the final-solution (as if there is nothing more to learn). We should just do exactly what they say, never think or ask questions or attempt to learn anything else, and follow them like the Pied-Piper for the rest of your career. We'll all be blind and bliss.

But we will also be left out in the cold when the rest of the scientific world wants to try to replicate it. When they point out deficiencies we will say - well you are not an expert so you don't understand or you didn't do it right. This is code for: you-don't-know-the-secret-pied-piper-polygraph-test-handshake, and this will NOT impress anyone.

We should ask to see the data whenever any has spectacular results that seem too good to be true. Or at least we should be able to expect to see the data if we actually wanted to (for example: if we had trouble sleeping)...Seriously now, scientists, like others, have been know to push their data and their results. That is why we have peer review, publication, and accountability requirements. Just look at cold fusion, the bell labs guy with the nano technology, and the recent fraud involving autism and vaccines.

OK, 'nuff for now.

Peace,

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 01-16-2011 10:27 AM            

quote:

---

However, the evidence seems to suggest that it is the decision rules and cutscores, and NOT the fancy linguistic precision of the test questions that is contributing to the change in criterion accuracy with multi-facet and multi-issue exams. Common criminals and psychopaths are linguistically imprecise creatures... It is unlikely that our nearly-OCD interest in semantic precision will actually connect with or impress their broken brains. Linguistic precision is for the truthful person - to attempt to eliminate causes for reaction other than deception regarding involvement in the factual behavioral concern described by the test question.

---

It's obvious that in our effort to cover all possible involvement and to satisfy our customers (agents/detectives/lawyers)we engage in "OCD" linguistic exercises such as:

Did you conspire with anyone to steal...?
Did you steal that....
Did you spend any of that stolen money?
Were you physically present when that money was stolen?

We assume we are covering all the bases and therefore preventing the thief from "rationalizing" his way out of the DI result. It seems, as Ray points out that we could just as easily use the single issue approach of

Were you involved in the theft of that money in any way? We then pre-test the definition of "involved in any way" to incorporate all of the possible ways.

If he passes, we have an accurate single issue test that can be used to support the examinee's lack of involvement. If he fails, we interrogate to determine his involvement, which is what we must do even if we use the multi-faceted approach.

In summary, I think we over complicate the process and reduce the efficacy of polygraph by asking multiple questions when a single question is often sufficient.

obviously, the multiple question approach has its place but it seems we should avoid it when possible to maintain the greatest accuracy.

posted 01-16-2011 11:51 AM            

quote:

---

Were you involved in the theft of that money in any way? We then pre-test the definition of "involved in any way" to incorporate all of the possible ways.

---

posted 02-07-2011 08:54 PM            

or

2.
PN - 0-10 sec
EDA - 0.5-12.9 sec
BP - 0-15 sec

posted 02-08-2011 10:33 AM            

Buy a new pistol or rifle and you might want to check the accuracy of the weapon and sight adjustment with some bench-rest shooting.

What is the best setting for the sights? Answer: the one that works the best.

You have the ability to adjust the measurement period with OSS-3. However, you probably should adjust it only if you have a really good reason for doing so and only if you know the answer to your question.

OSS-3 is a powerful tool, with many options. For most people it will be best to leave it in the default configuration and only select the type of examination, with occasional need to re-select the charts or questions.

Here is an example: My old Ford 4x4 truck has fixed-offset ball-joints on the front suspension. There is no adjustment that can be made to the caster or camber of the front wheels. Buy hey, its a Ford, and everyone knows that old Fords are what we call a "heads up" ride for which you will never take your eye off the road, or you will quickly wander off the road - and that is why they call it an off-road vehicle. And old Fords will always tend to shred front tires. Whadayawant? It is a tool for navigating rocks and ruts, and for hauling stuff for the house. Also, a really good driver, an expert, can drive the old Ford truck down the highway safely at the posted legal limit and remain accurately in a single lane.

Another example: a BMW, even an old one, will have ball joints that can be adjusted for caster and camber. Making careful adjustments can tune the vehicles performance during high speed driving.

Should you adjust it? No.

Just because you can doesn't mean you should. Most BMWs are not race-cars, they are used to commute to work, get groceries and to arrive at yoga class in style.

Why have the adjustment? Because that is what makes it a precision tool. Those who know what they are doing can have some fun.

The scoring window is an issue that has been studied, and for which we already have the answer from smart credible scientists.

The option to change the window is included only because there might be some need to study the issue further some day. Without access to the setting we could never study and never actually learn anything. Limiting options and restricting access to our data and tools may seem reasonable, but will actually keep us dumb and childlike in the long term.

The pace of learning has been a little slow in the polygraph profession. We have, at times, been accused of falling behind our sister professions, and now we risk not being taken seriously at all by the forensic science community.

The more I look at this the more I am convinced that the scoring window is a rather blunt issue - meaning it makes little, if any, noticeable difference. Of course you will notice a difference with individual questions and individual reactions no and then. You cannot change something and expect nothing to happen. The real concern is whether such a change will affect the overall profile of criterion accuracy with lots and lots of exams. It will not, and that is what science is concerned about. What happens with most exams most of the time.

Remember that justice is decided one case at a time, so we do also have to be concerned about individual cases. But if we have an examinee who is so odd that normative data does not apply, then we perhaps should be further evaluating the person's suitability for testing.

There is no science in unstandardized methods for which we make un-studies idiosyncratic adjustments to our scientific test methods just because someone is odd. Keep in mind that every psychopathy in the world thinks they are "special" and deserved special considerations of all kinds.

Kircher, Kristjianssson, Garder & Webb (2005) recommended to simply score to the end of the reaction.

Refering to your sample graphics, if you think that the little bit of non-reaction followed by more reaction was caused by the stimulus, then you should score it. If you think the little bit of non-reaction represents the end of the reaction and that the reaction after that is not due to the stimulus, then don't score it. Which is correct??? We don't really know - and that bothers those of us who cannot tolerate a little uncertainty and require hard fast rules in order to feel confident that we are smart and know what we are doing. (there is in fact, a lot of uncertainty in life). Anyway, we can only philosophize and opinionize and theorize about this until we study data. And we must remember that no matter how smart or experienced or degreed/pedigreed we are - most of our fancy hypothesis from our smartest scientists turn out not to actually work.

It is my option that the polygraph today is a very accurate test when conducted using techniques and principles that have been scientifically validated. It is so accurate that I believe that attempts to increase its accuracy with opinion and believe alone - without evidence of validity of some new hypothesis - will more likely damage its accuracy. Increased accuracy, for a highly accurate test, will not come cheaply or easily. We are going to have to work for it.

In the meantime, we should just keep it simple and do what the evidence tells us will work.

So Skar, my suggestion is this, why not study the question and learn the answer from data?

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 02-09-2011 02:39 AM            

First of all, of course, I am interesting in scoring windows which were used in the validation of OSS-3.

But some things that I have read make me doubtful about what is correct.

1. The annotation is in OSS3 spreadsheet:
"Measurements (Kircher and Raskin 1988; Raskin Kircher Honts and Horowits 1988)"

As I know scoring windows from "Kircher and Raskin 1988" are:
PN - 0-10 sec
EDA - 0-20 sec
BP - 0-20 sec

2. The settings are in the OSS3 spreadsheet:
P1 10
P2 10
E 15
C 15
F 15
M 15

3.

quote:

---

For training, OSS-3 uses whatever John Harris programmed into the tool he created for DACA(DoDPI) in 1994. There is no documentation, and he seems not to recall from memory. We have to assume he followed the instructions from Kircher and Raskin, 1988.
I suppose I could spend some time and try to figure it out further.
The best suggestion I have is to use Kircher's most recent recommendations. The effect of scoring window length, will be to assure adequate sensitivity to the signal of interest, while rejecting as much noise as possible. There are other considerats that are equally if not more important than window length - such as slope and artifact detection.

---

I have asked the question to clarify this issues for myself.

posted 02-09-2011 01:51 PM            

As I've said before we tend to chase our tails discussing and arguing over things that we think we understand, even when we already have the answer - instead of trudging into the complexities of the things we know we do not yet understand.

I do not know where you are getting that slope is important. Perhaps you could share that with us.

Artifact detection is a different matter.

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)

posted 02-09-2011 07:12 PM            

Thanks for clarifying this.

The slope referred to in the quoted section pertains to the identification of a change in slope value from negative to positive as the onset or starting point of a measured or observed reaction.

I was concerned that you might have been suggesting that the slope angle is somehow diagnostic. At this point, given our unknown about linearity of response and the various ways that signals are conditioned with various instruments, and considering what we know about diagnostic features in CQT PDD exams, the slope angle or rate of rise, does not seem to be a reliable diagnostic indicators. Neither does the recovery time or half-recovery time seem to be a worthwhile pursuit at this point.

Another tid-bit of interest... Kircher, Kristjiansson, Gardner $ Webb (2005) report that complexity is not diagnostic and is inversely correlated with deception. This means that it probably makes no difference - but it if it make a difference then we might actually be scoring it in the wrong direction.

So: here is my question: how is it that this type of thing happens - that our profession has seemed to endorse an idea that does not work?

.02

r

------------------
"Gentlemen, you can't fight in here. This is the war room."
--(Stanley Kubrick/Peter Sellers - Dr. Strangelove, 1964)